feat!: replace per-SP YugabyteDB with stock Postgres + Scylla#123
Open
rvagg wants to merge 2 commits into
Open
feat!: replace per-SP YugabyteDB with stock Postgres + Scylla#123rvagg wants to merge 2 commits into
rvagg wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the per-service-provider YugabyteDB dependency and replaces it with two stock, on-demand containers per SP: Postgres (HarmonyDB) and ScyllaDB (IndexStore). It also bumps the exported devnet-info.json schema to v2 to reflect the new database wiring and updates CLI/init, Docker utilities, CI, docs, and scenarios accordingly.
Changes:
- Replace the YugabyteDB start step with a new
DatabaseStepthat pulls/runspostgres:18+scylladb/scylla:2026.1per SP and wires Curio to both endpoints. - Remove Yugabyte-specific init/config/assets/build plumbing; adjust cleanup/log discovery to include stock-image DB containers by name.
- Update external API export to schema v2 (
database { postgres_port, scylla_port }) and update examples/docs/scenarios/CI for the new schema and DB access patterns.
Reviewed changes
Copilot reviewed 40 out of 41 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/port_allocator.rs | Update module docs to reflect Postgres/Scylla allocation instead of Yugabyte. |
| src/paths.rs | Remove Yugabyte volume-path helpers. |
| src/main.rs | Drop Yugabyte-related init CLI wiring. |
| src/main_app/version.rs | Stop printing default Yugabyte download URL. |
| src/main_app/command_handlers.rs | Remove Yugabyte args from init handler plumbing. |
| src/external_api/mod.rs | Export v2 schema types and bump schema version constant to 2. |
| src/external_api/export.rs | Build/export DevnetInfoV2, emitting per-SP database ports. |
| src/external_api/devnet_info.rs | Replace YugabyteInfo with DatabaseInfo in the schema model. |
| src/embedded_assets.rs | Remove embedded Yugabyte Dockerfile/volumes map. |
| src/docker/network.rs | Update network docs to reflect Curio + Postgres/Scylla. |
| src/docker/mod.rs | Remove Yugabyte build exports; add DB container naming exports. |
| src/docker/logs.rs | Include stock-image DB containers in log persistence and cleanup via name-matching. |
| src/docker/init.rs | Stop creating Yugabyte volume directories from embedded maps. |
| src/docker/core.rs | Improve image_exists() to handle tagged images and untagged repo checks. |
| src/docker/containers.rs | Add Postgres/Scylla container naming + DB container identification regex. |
| src/docker/build.rs | Remove special Yugabyte build flow; build only foc-devnet images. |
| src/constants.rs | Add stock DB image/port/credential constants; scope “foc-built images” list. |
| src/config.rs | Remove Yugabyte download URL from config defaults and struct. |
| src/commands/stop.rs | Add explicit per-SP DB container removal (and adjust stop ordering logic). |
| src/commands/status/running.rs | Report per-SP Postgres/Scylla containers in status running output. |
| src/commands/start/yugabyte/mod.rs | Remove Yugabyte start step implementation (deleted). |
| src/commands/start/mod.rs | Replace Yugabyte step with new database step in the startup sequence/epochs. |
| src/commands/start/database/mod.rs | New step: pull/run/verify Postgres + Scylla per SP. |
| src/commands/start/curio/mod.rs | Update step docs to refer to Postgres migrations (not Yugabyte setup). |
| src/commands/start/curio/db_setup.rs | Wire Curio env vars to Postgres + Scylla container hosts/ports. |
| src/commands/init/mod.rs | Remove Yugabyte artifact download; stage only proof params + build images. |
| src/commands/init/config.rs | Remove Yugabyte URL override handling from init config generation. |
| src/commands/init/artifacts.rs | Replace Yugabyte download/extract logic with lightweight artifact staging. |
| src/cli.rs | Remove --yugabyte-url / --yugabyte-archive init flags. |
| scripts/setup-scenarios-prerequisites.sh | Drop pyenv/Cassandra(cqlsh) install; rely on Scylla container tooling. |
| scenarios/test_caching_subsystem.py | Query proof-cache via docker exec <scylla> cqlsh instead of host ycql port + external cqlsh. |
| scenarios/synapse.py | Pin synapse-sdk commit that understands devnet-info schema v2 database block. |
| README_ADVANCED.md | Update docs for DB containers, schema, ports, and operational steps. |
| examples/read-devnet-info.js | Print DB ports from the new database block. |
| examples/devnet-schema.js | Update Zod schema to v2 and replace yugabyte with database. |
| docker/yugabyte/volumes_map.toml | Remove Yugabyte volume map (deleted). |
| docker/yugabyte/Dockerfile | Remove Yugabyte image build Dockerfile (deleted). |
| Cargo.toml | Remove downloader dependency now that Yugabyte download is gone. |
| Cargo.lock | Lockfile update consistent with dependency removal and resolver changes. |
| .github/workflows/ci_run.yml | Remove Yugabyte from cached images; adjust init and scenario prerequisites notes. |
| .github/copilot-instructions.md | Update project tech stack/docs to reflect Postgres + Scylla instead of Yugabyte. |
rvagg
force-pushed
the
rvagg/noyuga
branch
3 times, most recently
from
ae872b1 to
a41cdbd
Compare
Two stock containers per SP (postgres:18 HarmonyDB, scylladb/scylla
2026.1 IndexStore), pulled on demand. Yugabyte fully removed: image
build, tarball download, config field, init flags.
Start: 435s vs ~611-651s; migrations 1.5s vs 106-175s; ~145MiB/SP DB
RAM vs 1.28GiB. devnet-info.json schema v2: per-SP `yugabyte` block
replaced by `database` { postgres_port, scylla_port }.
Pins curio main 46a3640e (indexstore ALLOW FILTERING) and synapse-sdk
8ccff47 (loose v1/v2 schema); re-pin when tagged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two stock containers per SP (postgres:18 HarmonyDB, scylladb/scylla 2026.1 IndexStore), pulled on demand. Yugabyte fully removed: image build, tarball download, config field, init flags.
Start: 435s vs ~611-651s; migrations 1.5s vs 106-175s; ~145MiB/SP DB RAM vs 1.28GiB. devnet-info.json schema v2: per-SP
yugabyteblock replaced bydatabase{ postgres_port, scylla_port }.Pins curio main 46a3640e (indexstore ALLOW FILTERING) and synapse-sdk 8ccff47 (loose v1/v2 schema); re-pin when tagged.
(tbh I'm not really expecting someone to review all of this, will let copilot and CI do its job, don't let me stop you reviewing it if you want though!)